Deep Learning based Car Identification

- Automotive, Surveillance, Object Detection & Localisation

Project By:
Premjeet Kumar https://www.linkedin.com/in/premjeet-kumar/
Hari Samynaath S https://www.linkedin.com/in/harinaathan/
Veena Raju https://www.linkedin.com/in/veena-raju-1b16b513b/
Javed Bhai https://www.linkedin.com/in/javedbhai/
Surabhi Joshi https://www.linkedin.com/in/surabhi-joshi-4452788/

Project For:
Captstone project for Post Graduate Program in Artificial Intelligence and Machine Learning
with GreatLakes & Texas McCombs School of Business, The University of Texas at Austin

CONTEXT:
Computer vision can be used to automate supervision and generate action appropriate action trigger if the event is predicted from the image of interest. For example a car moving on the road can be easily identi ied by a camera as make of the car, type, colour, number plates etc.

DATA DESCRIPTION:
The Cars dataset contains 16,185 images of 196 classes of cars. The data is split into 8,144 training images and 8,041 testing images, where each class has been split roughly in a 50-50 split. Classes are typically at the level of Make, Model, Year, e.g. 2012 Tesla Model S or 2012 BMW M3 coupe.

‣ Train Images: Consists of real images of cars as per the make and year of the car.
‣ Test Images: Consists of real images of cars as per the make and year of the car.
‣ Train Annotation: Consists of bounding box region for training images.
‣ Test Annotation: Consists of bounding box region for testing images.

MILESTONE 1:

‣ Step 1: Import the data
‣ Step 2: Map training and testing images to its classes.
‣ Step 3: Map training and testing images to its annotations.
‣ Step 4: Display images with bounding box
‣ Output: Images mapped to its class and annotation ready to be used for deep learning

Step 1: Import the data

The following libraries and packages are used for reading the csv files, processing the data and visualizing the data / images

Read CSV : Read the car names and make and change the name of the column to "fullNames"

Read Word Count :

By understanding the length of the words, we can plan to split the OEM, Model, Type and Year part from the fullName

The following help us to understand that most of the entries are of 4 words and around 6 of them are having word length as 7

Visualize

Print the entries with 7 words

Cleanup Process

Remove '/' character from the entries for further spliting them to OEM, Model, Type and Year values

Split the data

Extract the OEM, Year information from the entries. Check the words in between belongs to OEM or Model name

Analysis of string data

This will help us to understand the combinations of OEM column value with the multiple values in chk column. This will help to develop a logic based on the pattern in which the OEM and Model names are placed in the entry names

Analysis of the string data after the first word

Get only the entries where the entry in chk is 1 list item or unique item.

eg; AM has only one unique value in chk column ie; [General]

whereas Acura has multiple words in chk column [RL, TL, TSX, Integra, ZDX].

This will help to conclude that any entries with only 1 value in 'chk' column is part of OEM name and not Model name

Extract OEM Values

As per our findings above, we can extract the OEM correctly from the entries. We can add a condition that, for those OEM names that contains 'AM' or 'Aston' or 'Land', we can get the second word also as part of the OEM name

Extract Model name for a single entry

Extract the model name for the 10th item in the dataframe to check whether the conditions on OEM and YEAR will get the Model names seperate

Extract Model Name

Extract the model name from the fullNames for all entries using the logic tried out above

Separate Model and Type

Check for the length of the Model name for all entries in dataframe. This helps to build the logic around the way the Model name and Type can be separated

Print the length of strings

Print the values from dataframe for word count ==5

Print the values from dataframe for word count ==4

Print the values from dataframe for word count ==3

Pattern to extract the Model and Type

Note that Cab & Van comes with 2 word coach type. Othewise, almost every other words are part of model name only. Hence we can separate the coach type

Extract the type value to a new column "Type"

Get unique values in Type column

Findings

We can update the TYPE accordingly

Update Model names for all entries, excluding the Type information

Combine the OEM names, Model Names and Type

Combine the values using "_" if the values in Model name or Type is a list else, display as such

Finalize the schema / structure of Master Data

Print Summary Information

Print unique values count of each column

Visualization - Data Distribution

BAR CHART

A bar chart is used to show the distribution of data points so that we can perform a comparison of metric values across different subgroups. From the chart, we can see which groups are highest or most common, and how other group compare against the others.

Number of Models available under each OEM value

Distribution : Total number of OEMs - 49

The Chevrolet is having 11% of contribution for the various models

The Ram, Porsche, AM General, Jaguar, smart are few models that are contributing only 0.05% of the models

This help to understand about the imbalance of data for OEM and Models in the dataset

Number of Types available under each Model value

Distribution : Total number of Sedan - 46

The Mitsubishi-Sedan is having 23% of contribution for the various models-types

The Club Cab, Wagon Van, Passenger Van are few models types that are contributing only 0.05% of the model-types

This help to understand about the imbalance of data for Model and Types in the dataset

Number of Models available under each Year

Distribution : Total number of Models in the year 2012 - 117

The 2012 is having 60% of contribution for the various models for the years

The models from 1991-2000 is having the least contribution of models counting to 0.05%

This help to understand about the imbalance of data for Models and Year in the dataset

Observations from the Visualization - Bias

There are imbalances in the dataset might create bias in the model's capabilities

Bias Type : Data Collection Bias

Description : Bias introduced by the selection of individuals, groups (eg; OEM, Model, Type, Year) in such a way that proper randomization is not achieved. This will fail to ensure that the sample obtained is representative of the population intended to be analyzed.

We could clearly understand that the contribution by few OEM , few Models and selected years are more than others. This will lead to the model being biased towards some OEM-Model-Year cars.

Eg; the model learns more of "Chevrolet" OEM than others. will learn "Sedan" Model than others and 2012 Year Model than others

To reconfirm the findings, let us also read the image data files in to our notebook, and review the distribution once again

Step 2: Map training and testing images to its classes.

Read Images for test and train

Train Images

Test Images

Compute image size

Store the image size of height and width in new column called "pixels"

Print Image dimensions

This will help to visualize the dimensions of the images in range

Resizing Images

Resizing images is a critical preprocessing step in computer vision. Machine Learning models  train faster on smaller images and they need images of same size as input.

Some of the Best Practices

1. To decide on what should be the size of the images, a good strategy is to employ 
progressive  resizing. eg; we can start with all images resized to the smallest one.

2. Progressive resizing will train an initial model with very small input images and gauge 
performance. We can use those weights as the starting point for the next model with larger 
input images.

3. Downsizing larger images to match the size of smaller images is often a better bet 
than increasing the size of small images to be larger.

4. In general, it is safer to maintain the raw image aspect ratio and resize 
proportionally.

5. Make use of image resizing methods like interpolation so that the resized images 
do not lose much of their perceptual character.

Image Interpolation

Image interpolation occurs when you resize or distort your image from one pixel grid to 
another. There are two types of interpolation. 

1. Adaptive : Adaptive methods change depending on what they are interpolating
2. Non-adaptive : Non-adaptive methods treat all pixels equally.

Higher-Order Interpolation techniques like Spline and Sinc are computationally costly, 
where as Nearest Neighbor, bilinear are computationally less expensive.

How to best resize the given images?

We can optimally learn representations of images for a given resolution by consistently 
improving the performance of the common vision models. We can use bilinear interpolation with learnable image resizing module using keras will help to acheive this

Ref: https://keras.io/examples/vision/learnable_resizer/

Initial Image Size

Based on above review, we shall restrict the image size fed to the network at 50x50 pixels, so as not to detoriate lower resolution images and thus affect model capabilities

Step 3: Map training and testing images to its annotations

Read Bounding box and annotations

Having connected to the images directories, lets also add the annotations, and add the bounding boxes to the images

Merge all information of images, annotations, bounding box to single DataFrame

Merge OEM,MODEL,Type,Year with the above dataframe

Validate data for any mismatch during merging

After doing the cross merged and synced with "Train/Test Annotations.csv", "Car names and make.csv" and the images in the "Train/Test images folders", it is found to have no mismatch of information

Cleanup - Unwanted columns

Remove unwanted columns and make the dataframe more readable

Check for null values

Step 4: Display images with bounding box

Visualization - Images with bounding box and annotations - 5 Nos

Visualize

Number of Images per OEM

Distribution : Total number of OEM - Chevrolet - 905

The Chevrolet is having 60% of contribution for the images in train dataset

The Maybach is having the least contribution of images in train dataset, as 0.03%

This help to understand about the imbalance of data for OEM-images dataset

Visualize

Number of Images per OEM-Type

Distribution : Total number of Mitsubishi-Sedan - 1907

The Mitsubishi-Sedan is having 26% of contribution for the images in train dataset

The Express Van is having the least contribution of images in train dataset, as 0.04%

This help to understand about the imbalance of data for Model-images dataset

Visualize

Number of Images per Year

Distribution : Total number of images for year 2012 - 4818

The 2012 is having 60% of contribution for the images in train dataset

The 1997-1999 is having the least contribution of images in train dataset, as 0.05%

This help to understand about the imbalance of data for Year-images dataset

Distribution of Support

The box plot shows how the distribution of images are there for each class / category

Print Cross distribution of the Type from each OEM

Print distribution of support images

The combinations are only within 40~50 images each, whith few extreme counts

Print Combination with more than 50 image (Average)

Print Combination with less than 35 image

Conclusion

1. All the data preprocessing & compilation have been completed so far
2. The data were imported and mapped against their respective classses & annotations
3. Comprehensive dataframes for training & testing datasets were created and could be used 
with generators for Deep Learning Networks

MILESTONE 1 COMPLETE

for future codes, import the following dataframes as final output from this Milestone1

use following code to invoke & import data
from Milestone1 import trainDF, testDF;